Image processing method and electronic device supporting the same

ABSTRACT

An image processing method is provided, including the operations of collecting a first motion picture, extracting spatial information about at least a part of frames forming the first motion picture, and reflecting the extracted spatial information to generation of a 3D view corresponding to the first motion picture.

CLAIM OF PRIORITY

This application claims the benefit under 35 U.S.C. §119(a) of a Koreanpatent application filed on Feb. 27, 2015 in the Korean IntellectualProperty Office and assigned Serial number 10-2015-0028646, the entiredisclosure of which is hereby incorporated by reference.

TECHNICAL FIELD

The present disclosure relates to an image processing method forgenerating 3D views using 2D picture data and an electronic devicesupporting the method.

BACKGROUND

Electronic devices, e.g., smart phones, tablet computers, and digitalcameras, can provide various functions such as taking photos or motionpictures, media outputs, and so on. In taking photos or motion pictures,those electronic devices may diverse provide options therefor.Generating 3D views from original 2D pictures (original pictures) wouldtake a considerably large amount of time. The original pictures would bephotographed and stored in a memory of a user terminal. The 3D viewwould be generated from the stored 2D pictures. The original pictureswould be transmitted to an external server and the external server wouldgenerate a 3D view from the original pictures which would be receivedfrom the user terminal. The external server can then transmit thegenerated 3D view to the user terminal and the user terminal outputs thereceived 3D view. The foregoing would make real-time error checking ofthe 3D view generating while photographing the 2D pictures unfeasible.

SUMMARY

Aspects of the present disclosure are to address at least theabove-mentioned problems and/or disadvantages and to provide at leastthe advantages described below. Accordingly, an aspect of the presentdisclosure is to provide an image processing method for generating 3Dviews using 2D picture data and an electronic device supporting themethod. In accordance with an aspect of the present disclosure,

According to various embodiments of the present disclosure, it may beallowable for an electronic device to shorten a picture analyzing timeby generating 3D information based on an original picture and anadditional analyzing picture.

According to various embodiments of the present disclosure, it may bealso allowable for an electronic device to improve the photographingaccuracy of 3D view generation by a mode of photographing an originalpicture, based on spatial information extracted from the originalpicture and an additional analyzing picture.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram illustrating an electronic device according tovarious embodiments.

FIG. 2 is a flow chart showing a process of 3D view generation accordingto various embodiments.

FIG. 3A and FIG. 3B are exemplarily diagrams illustrating a process ofcollecting first picture information according to various embodiments.

FIG. 4A is an exemplary diagram illustrating a process of collectingsecond picture information from first picture information according tovarious embodiments.

FIG. 4B is an exemplary diagram illustrating a process of generating a3D view using spatial information and a stored first picture accordingto various embodiments.

FIG. 4C is an exemplary diagram illustrating a method of generating a 3Dview in real time from a 2D picture and spatial information.

FIG. 4D is an exemplary diagram illustrating a method of generating a 3Dview through a background change using a second motion picture andspatial information.

FIG. 5A and FIG. 5B are exemplary diagrams illustrating processes ofsampling a second motion picture from a first motion picture accordingto various embodiments.

FIG. 6 illustrates a process of correcting a first motion picture usinginformation extracted from a second motion picture according to variousembodiments.

FIG. 7 illustrates a process of correcting a first motion picture basedon feature information of a second motion picture according to variousembodiments.

FIG. 8 is a flow chart showing a process of mode change based on asecond motion picture according to various embodiments.

FIG. 9 is a flow chart showing a process of photographing mode changeaccording to various embodiments.

FIG. 10 is a flow chart showing an object centering process according tovarious embodiments.

FIG. 11 is an exemplary diagram illustrating screens for an objectcentering process according to various embodiments.

FIG. 12A and FIG. 12B are exemplary diagrams illustrating screens using3D views according to various embodiments.

FIG. 13 is a block diagram illustrating an electronic device accordingto various embodiments.

FIG. 14 is a block diagram illustrating a program module according tovarious embodiments.

DETAILED DESCRIPTION

Hereinafter, various embodiments of the present disclosure will bedescribed in conjunction with the accompanying drawings. Variousembodiments described herein, however, may not be intentionally confinedin specific embodiments, but should be construed as including diversemodifications, equivalents, and/or alternatives. With respect to thedescriptions of the drawings, like reference numerals refer to likeelements.

The terms “have”, “may have”, “include”, “may include”, “comprise”, or“may comprise” used herein indicate existence of corresponding features(e.g., numerical values, functions, operations, or components) but doesnot exclude other features.

As used herein, the terms “A or B”, “at least one of A or/and B”, or“one or more of A or/and B” may include all allowable combinations whichare enumerated together. For example, the terms “A or B”, “at least oneof A and B”, or “at least one of A or B” may indicate all cases of: (1)including at least one A, (2) including at least one B, or (3) includingboth at least one A, and at least one B.

As used herein, the terms such as “1st”, “2nd”, “first”, “second”, andthe like may be used to qualify various elements regardless of theirorder and/or priority, simply differentiating one from another, but donot limit those elements thereto. For example, both a first user deviceand a second user device indicate different user devices. For example, afirst element may be referred to as a second element and vice versawithout departing from the scope of the present disclosure.

As used herein, if one element (e.g., a first element) is referred to asbeing “operatively or communicatively connected with/to” or “connectedwith/to” another element (e.g., a second element), it should beunderstood that the former may be directly coupled with the latter, orconnected with the latter via an intervening element (e.g., a thirdelement). Otherwise, it will be understood that if one element isreferred to as being “directly coupled with/to” or “directly connectedwith/to” with another element, it may be understood that there is nointervening element (e.g., a third element) existing between them.

In the description or claims, the term “configured to” (or “set to”) maybe changeable with other implicative meanings such as “suitable for”,“having the capacity to”, “designed to”, “adapted to”, “made to”, or“capable of”, and may not simply indicate “specifically designed to”.Alternatively, in some circumstances, a term “a device configured to”may indicate that the device “may do” something together with otherdevices or components. For instance, a term “a processor configured to(or set to) perform A, B, and C” may indicate a generic-purposeprocessor (e.g., CPU or application processor) capable of performing itsrelevant operations by executing one or more software or programs whichis stored in an exclusive processor (e.g., embedded processor), which isprepared for the operations, or in a memory.

The terms used in this specification are just used to describe variousembodiments of the present disclosure and may not be intended to limitthe scope of the present disclosure. The terms of a singular form mayinclude plural forms unless otherwise specified. Unless otherwisedefined herein, all the terms used herein, which include technical orscientific terms, may have the same meaning that is generally understoodby a person skilled in the art. It will be further understood thatterms, which are defined in a dictionary and commonly used, should alsobe interpreted as is customary in the relevantly related art and not inan idealized or overly formal detect unless expressly so defined hereinin various embodiments of the present disclosure. In some cases, termseven defined in the specification may not be understood as excludingembodiments of the present disclosure.

Hereinafter, an electronic device according to various embodiments willbe described in conjunction with the accompanying drawings. Indescription for various embodiments, the term “user” may refer to aperson using an electronic device or a device (e.g., an artificialintelligent electronic device) using an electronic device.

FIG. 1 is a block diagram illustrating an electronic device according tovarious embodiments.

Referring to FIG. 1, an electronic device 101 may include a bus 110, aprocessor 120, a memory 130, a sensor module 140, an input/outputinterface 150, a display 160, a communication interface 170, a cameramodule 180, and a 3D conversion module 190. In some embodiments, theelectronic device 101 might not include at least one of the foregoingelements or may further include another element therein.

The bus 110, for example, may include a circuit for connecting theprocessor 120, memory 130, sensor module 140, input/output interface150, display 160, communication interface 170, camera module 180, andthe 3D conversion module 190 to each other and relaying communication(control messages and/or data) therebetween.

The processor 120 may include at least one or more of a centralprocessing unit (CPU), an application processor (AP), or a communicationprocessor (CP). The processor 120, for example, may execute computationor data operation for control and/or communication of other elements ofat least one of the electronic device 101. In various embodiments, theprocessor 120 may process information which is collected through thesensor module 140. The processor 120 may determine information about amoving direction, a moving distance, an acceleration of the electronicdevice 101, based on information collected through the sensor module140. Information calculated through the processor 120 may be provided tothe 3D conversion module 190.

The memory 130 may include a volatile and/or nonvolatile memory. Thememory 130 may store, for example, instructions or data which areinvolved in at least one of other elements in the electronic device 101.According to an embodiment, the memory 130 may store software and/or aprogram

The sensor module 140, for example, may measure a physical quantity ormay detect an operation state of the electronic device 101, and mayconvert the measured or detected information into an electric signal.The sensor module 140, for example, may include a gesture sensor, a gyrosensor, an acceleration sensor, a grip sensor, a proximity sensor, abiometric sensor, an illuminance sensor, among other things.

According to various embodiments, the sensor module 140 may collectinformation about a moving direction, a moving speed, a location of theelectronic device 101. The corrected information may be provided to theprocessor 120 or the 3D conversion module 190.

The input/output interface 150 may act, for example, as an interfacecapable of transferring instructions or data, which are input from auser or another external device, to another element (or other elements)of the electronic device 101. Additionally, the input/output interface150 may output instructions or data, which are received from anotherelement (or other elements) of the electronic device 101, to a user oranother external device.

The display 160 may include, for example, a liquid crystal display(LCD), a light emitting diode (LED), an organic LED (OLED) display, amicroelectromechanical system (MEMS) display, or an electronic paper.The display 160 may display, for example, diverse contents (e.g., text,picture, video, icon, or symbol) to a user. The display 160 may includea touch screen, and for example receive an input of touch, gesture,approach, or hovering which is made by using an electronic pen or a partof a user's body.

The communication interface 170 may set, for example, a communicationcondition between the electronic device 101 and an external electronicdevice (e.g., a first external electronic device 102, a second externalelectronic device 104, or a server 106). For example, the communicationinterface 170 may communicate with an external electronic device (e.g.,the second external electronic device 104 or the server system 106) inconnection with a network 162 through wireless communication or wiredcommunication.

The camera module 180 may take a photograph or picture. In variousembodiments, 2D images (for purposes of this document, the term “image”shall be a general term that includes, but is not limited to, a “frame”,“frames” or a portion of a “frame”) collected through the camera module180 may be used for generating a 3D view. A user may use the cameramodule 180 to photograph an object which is to be converted into a 3Dview. A user may use the camera module 180 to photograph an object whilemoving along a specific path (e.g., a circular path) around the object.From the photographed picture, spatial information, such as features anddepths, may be extracted through the 3D conversion module 190 and thenmay be converted into a 3D view. In various embodiments, the spatialinformation may include at least a part of the following information:(1) relative coordinate information (X-Y-Z locations and yaw-pitch-rolldirections); (2) depths respective to features (depth may be an intervalbetween a camera and a part (e.g., a feature) of a shape of an object);and (3) relative conversion information between photographed picturesand estimated relative camera moving information by the relativeconversion information.

The 3D conversion module 190 may process a picture, which isphotographed through the camera module 180, and then may generate a 3Dview. The 3D conversion module 190 may extract spatial information aboutfeatures and depths and then may generate a 3D view for an object. Invarious embodiments, if the camera module 180 begins to photograph anobject, the 3D conversion module 190 may generate an analyzing picture,which is scaled down or sampled, based on a photographed originalpicture. The 3D conversion module 190 may simultaneously execute aprocess of extracting information, which is necessary for generating a3D view, from the analyzing picture, and a process of photographing theoriginal picture.

According to various embodiments, the 3D conversion module 190 mayinclude a spatial-information extraction part 191, and a 3D-viewgeneration part 192. The spatial-information extraction part 191 mayextract spatial information such as features and depths. Features may bedetermined based on contrast/color values of pictures. Depths may berelative intervals between features and the electronic device 101. The3D-view generation part 192 may generate a 3D view to an object based onspatial information such as extracted features and depths.

Elements illustrated in FIG. 1 may be functionally classified and maynot be restrictive hereto. For example, an operation processed in the 3Dconversion module may be even performed through the processor 120.Additionally, the 3D conversion module 130 may be even included in aform of internal module of the processor 120.

FIG. 2 is a flow chart showing a process of 3D view generation accordingto various embodiments.

Referring to FIG. 2, at operation 210, a camera module 180 begins tophotograph an object and to collect a first motion picture therefrom.For example, a user may execute a 3D camera application (hereinafter,curtly referred to as ‘3D app’) to photograph an object. The 3D app maycall the camera module 180 and may collect the first motion picture.After the beginning of the photographing, a user may photograph areas ofan object, which are to be converted into a 3D view, while moving aroundthe object in a circular form (e.g., while rotating 360° on the centerof the object). The collected first picture may be stored in a buffer orthe memory 130. The first motion picture may include a collection ofstill frames taken during regular periodic intervals (such as every 1/20of one second) over a continuous period of time from continuouspositions.

At operation 220, the 3D conversion module 190 may generate a secondmotion picture which is composed of at least a part of the framesforming the first motion picture (the second motion picture may not beframes which are continuously photographed, but may be data itself of aplurality of 2D images sampled from the first motion picture). Thesecond motion picture may be an analyzing picture which is scaled downor sampled from the first motion picture under a specific condition. Invarious embodiments, the 3D conversion module 190 may execute a samplingprocess after reducing a data processing capacity by scaling the firstmotion picture down.

According to various embodiments, the second motion picture might not becollected after completing the photographing of the first motionpicture, but may be collected through a background processing while thecamera module 180 is continuously collecting the first motion picture.For example, if the photographing begins to take the first motionpicture, the 3D conversion module 190 may scale down the first motionpicture every specific time interval (e.g., 0.1 sec) and then maycollect the second motion picture by sampling the scaled-down picture.Various sampling modes will be described later in conjunction with FIG.5.

At operation 230, the 3D conversion module 190 may collect spatialinformation for the second motion picture, such as features, depths, andphotographing locations. The 3D conversion module 190 may not directlyextract spatial information of the first motion picture which is theoriginal picture, but may extract spatial information from the secondmotion picture which is an analyzing picture sampled from the firstmotion picture. In this manner, the 3D conversion module 190 may extractinformation necessary for 3D conversion, using the scaled-down orsampled second picture, without waiting for an end of the photographingof the first motion picture (including a completion time of encoding amotion picture with the first motion picture which is being continuouslyphotographed). Additionally, the 3D conversion module 190 may reduce anamount of calculated data, through the scaling-down or sampling, and mayextract spatial information while the first motion picture isphotographing.

At operation 240, the 3D conversion module 190 may reflect the spatialinformation of the second motion picture into a 3D-view generationprocess in correspondence with the first motion picture. For example,the 3D conversion module 190 may reflect spatial information of thesecond motion picture to an additionally continuing photographing forthe first motion picture, or may utilize the spatial information tosimplify a 3D-view generation process for the photographed firstpicture.

In various embodiments, the 3D conversion module 190 may generate a 3Dview for the second motion picture and may provide the 3D view to auser. The user may additionally photograph the first motion picturewhile confirming the 3D view based on analyzing picture.

FIGS. 3A and 3B are exemplarily diagrams illustrating a process ofcollecting first picture information according to various embodiments.

Referring to FIG. 3A, an electronic device 101 may photograph a firstmotion picture through a camera module 180. A user may collect the firstmotion picture while gradually moving the electronic device 101 along aspecific direction (e.g., from the left to the right) around an object310. The first motion picture may be stored in a buffer or the memory130. Although FIG. 3A is exemplarily illustrated as photographing theobject 310 while a user is moving from the left to the right about thecenter of the object 310, various embodiments of the present disclosuremay not be restrictive hereto. According to a mode to be set, the firstmotion picture may be even collected by photographing the object in aright-left direction or an up-down direction.

In the case that the electronic device 101 photographs the object 310 ata first location (e.g., the front direction of the object 310), a firstimage 310 a may be collected. In the case that the electronic device 101photographs the object 310 at a second location (e.g., a side directionof the object 310), a second image 310 b may be collected. The imagesare taken by photographing the object 310 in different angles and mayinclude different features each other thereon.

Additionally, the electronic device 101 may use the first image 310 aand the second image 310 b, which are photographed at differentlocations, to detect spatial locations respective to features.Additionally, the electronic device 101 may even detect a relativedirection between a camera and an object, and positional informationthereabout.

In various embodiments, the electronic device 101 may compare aplurality of images each other to analyze relations of features and toextract positional information (homography) between a camera and anobject. Additionally, the electronic device 101 may use parallaxinformation, which arises from a camera motion, even to extract depthinformation (depth from motion) of an abject.

According to various embodiments, in the case that spatial informationcannot be confirmed only by pictures, the electronic device 101 maycollect the spatial information through diverse sensors included in asensor module 140. For example, in the case that features cannot beextracted, the electronic device 101 may use an acceleration sensor anda GPS sensor to extract information about moving directions, movingspeeds, and locations from places at which frames are respectivelyphotographed.

FIG. 3B illustrates a photographed picture at a first location accordingto various embodiments.

Referring to FIG. 3B, a picture 310 a may include a multiplicity offeatures (point cloud). The front figure image, e.g., the picture 310 a,may have relatively many features. Contrarily, the back figure image mayhave relatively small features.

In the case of photographing an object while moving the left to theright as illustrated in FIG. 3A, the left area of the picture 310 a maybe a loss field 311 and the right area may be a generation field 312.The loss field 311 may include a relatively large number of features atthe beginning time of photographing. In the case that an electronicdevice 101 is moving from the left to the right after the beginning ofphotographing, the loss field 311 may decrease in the number of featuresfor a specific part of the object. Differently, the generation field 312may include a relatively small number of features for a specific part ofthe object, but may increase in the number of features to the specificpart during the photographing.

The 3D conversion module 190 may extract spatial information of a secondmotion picture instead of extracting spatial information of features anddepths for all frames of the first motion picture. In the case ofprocessing all of spatial information for an original picture, an amountof data may be massive. Furthermore, it may take a long processing timeand a user may spend a long time for confirming a 3D result. To overcomethese problems, the 3D conversion module 190 may shorten a time forconfirming a result of 3D conversion by first extracting spatialinformation for a sampled second picture and then by reflecting thespatial information to a photographing process of the first motionpicture or a process of the 3D conversion.

FIG. 4A is an exemplary diagram illustrating a process of collectingsecond picture information from first picture information according tovarious embodiments.

Referring to FIG. 4A, a camera module 180 may begin to collect a firstmotion picture 410. A 3D conversion module 190 may collect a secondmotion picture, which is scaled down or sampled from the first motionpicture 410, in addition to continuously photographing and storing thefirst motion picture 410. Hereinbelow, the description will be focusedon a sampling after scaling the first motion picture 410 down, whereasembodiments of the present disclosure may not be restrictive hereto.

The 3D conversion module 190 may scale the collected the first picture410 to generate the second motion picture 420. After then, the 3Dconversion module 190 may generate a sampled second picture 430 inaccordance with a specific condition. For example, the sampled secondpicture 430 may be a subset of frames from the second motion picturethat are collected at a lower sampling frequency, or higher samplinginterval.

In addition to the photographing of the first motion picture 410, the 3Dconversion module 190 may collect spatial information 440 a, such asfeatures and depths of frames forming the second motion picture 430,through a background work and then may store the collected spatialinformation in a buffer or memory.

According to various embodiments, the 3D conversion module 190 mayextract spatial information 440 b for the rest of the frames 450 of thesecond motion picture, which are not sampled, based on the generatedspatial information 440 a. The 3D conversion module 190 may processother frames based on the preliminarily calculated and stored spatialinformation 440 a, thus shortening a calculation time. The 3D conversionmodule 190 may store spatial information 450 b, which is involved in allframes of the scaled-down first picture, in a buffer or memory. Invarious embodiments, an operation for extracting the spatial information440 b for the rest frame 450 may be performed while the first motionpicture is photographing, or may be performed through a backgroundprocessing while the first motion picture is playing.

After completing the photographing of the first motion picture 410, the3D conversion module 190 may generate a picture 411 which is transformedinto a motion picture format (e.g., MP4) or a still picture format(e.g., PNG), and may store the picture 411 together with spatialinformation 440 a which is generated therefrom.

In various embodiments, in the case that a user selects the stored firstpicture 411 for 3D view generation, the 3D conversion module 190 maygenerate a 3D view based on spatial information 412 and spatialinformation 440 b which are stored together.

FIG. 4B is an exemplary diagram illustrating a process of generating a3D view using spatial information and a stored first picture accordingto various embodiments.

Referring to FIG. 4B, a 3D conversion module 190 may generate a 3D view462 in real time from a first motion picture 460 and spatial information460 a. The 3D conversion module 190 may extract frame-respective spatialinformation and a 2D picture from a file of the first motion picture 460and may decode the picture. A decoded picture may be used for extractingmore spatial information through additional calculation. For example,the additional calculation may be performed to further extractinformation about a frame 461, which is not included in the existingspatial information, as illustrated in FIG. 4B, or may be performed evento improve the accuracy of 3D spatial information (e.g., to expand theaccuracy from 3 Degree-Of-Freedom (DOF) information to 6-DOFinformation). The additional calculation may be used with preliminarilystored spatial information and a decoded picture.

Extracted spatial information may be applied to a decoded picture togenerate and display a 3D view 462. A direction or location of the 3Dview may be determined in accordance with a direction or location of auser terminal during a playback, a gesture input of a user, or a signalgenerated by remote control from another device.

FIG. 4C is an exemplary diagram illustrating a method of generating a 3Dview in real time from a 2D picture and spatial information.

Referring to FIG. 4C, a 3D conversion module 190 may classify features,which are detected from a picture, and spatial information, whichcorresponds to an object 470, among spatial information relevant to thefeatures. The 3D conversion module 190 may set an area (i.e., dottedarea) formed of features corresponding to the object 470 which isclassified, and may generate 3D view information to make the areauniform in at least one of location, size, or direction. The 3Dconversion module 190 may use such obtained information about adirection of the object 470 and may select a picture of the object 470,which is rotated by a specific angle from the same input, even togenerate 3D view information.

According to various embodiments, in the case of additionally usinginformation about an area (e.g., visage part) where principal featuresare disposed, the 3D conversion module 190 may set a size and locationof the area to be uniform on the same condition even to other pictures.For example, in the case of setting a face or visage to occupy thescreen center and ⅓ of the total view angle in a picture which isrotatably photographed on the center with the visage, the 3D conversionmodule 190 may provide the same 3D view even though the photographingwas taken in different intervals.

FIG. 4D is an exemplary diagram illustrating a method of generating a 3Dview through a background change using a second motion picture andspatial information.

Referring to FIG. 4D, among features, which are detected from a picture,and spatial information involved in the features, a 3D conversion module190 may classify spatial information corresponding to an object 480, andspatial information corresponding to a background 481 surrounding thecircumference of the object 480. The 3D conversion module 190 may set anarea (dotted area) formed of features corresponding to an object 480which is classified, may retain the uniformity of the area in location,size, or direction, and may utilize spatial information of thebackground 481 to generate a 3D view in which the background rotates.The 3D view with the rotating background 481 may provide a user with avisible effect different from a 3D view in which the object 480 rotates.

FIGS. 5A and 5B are exemplary diagrams illustrating processes ofsampling a second motion picture from a first motion picture accordingto various embodiments.

FIG. 5A illustrates a process of sampling the second motion picturebased on a moving path while an electronic device 101 is photographingan object.

Referring to FIG. 5A, a 3D conversion module 190 may generate the secondmotion picture based on a moving path of the electronic device 101. If acamera module begins to photograph an object for the first motionpicture, the 3D conversion module 190 may sample frames, which form thesecond motion picture, at points where a photographing path (ideal path)510 suitable for generating a 3D view intersects or adjoins an actualpath 520 of the electronic device 101.

The ideal path 510 may be a preliminarily designated path or a pathdetermined by analyzing early frames of the first motion picture. Theactual path 520 may be a path identified through a motion sensorincluded in the electronic device 101.

For example, sampled frames 511, 512, and 513 may be determined asframes forming the second motion picture in the case that the ideal path510 intersects or adjoins the actual path 520. Otherwise, other framessuch as frames 521 and 522 may not be included in the sampled secondpicture because the ideal path 510 is different more than a specificvalue from the actual path 520.

According to various embodiments, even in the case that the ideal path520 intersects or adjoins the actual path 520, some frames may beexcluded by a specific condition (e.g., sampling interval). For example,a frame 523 corresponds to the intersection case of the ideal path 510and the actual path 520, but may be excluded from the second motionpicture in consideration of a sampling interval with the frame 513 whichhas been already sampled.

According to various embodiments, parameters for correcting other frames(e.g., the frames 521, 522, and 523) of the first motion picture, whichare not sampled into the sampled second picture, may be determined basedon the sampled frames 511 to 513. Information about the correctionparameters may be provided as illustrated in FIG. 6.

FIG. 5B illustrates a process of collecting the sampled second pictureby cropping a part of a frame.

Referring to FIG. 5B, if a camera module 180 begins to photograph anobject for the first motion picture, a 3D conversion module 190 mayremove the reset area except a reference area 530 from each frame, basedon identification information of a motion sensor. The reference area 530may be set as most including the object or features.

For example, for frames 531 to 533, parts of frames 531 a, 532 a, and533 a, which are included in the reference area 530, may be included inthe second motion picture and the remaining portions 531 b, 532 b, 533 cmay be cropped and removed therefrom. The 3D conversion module 190 mayremove a circumferential area, which does not include an object, toreduce an amount of data to be processed.

The sampling modes illustrated in FIGS. 5A and 5B are exemplary proposedand various embodiments of the present disclosure may not be restrictivehereto. For example, the 3D conversion module 190 may sample the framesbased on positional differences of the electronic device 101photographing the frames, or based in differences of features includedin the frames.

According to various embodiments, the respective sampling modes may beeven coincidentally performed in combination. For example, asillustrated in FIG. 5A, after the sampling based on the moving path, apart of a sampled frame may be cropped to reduce an amount of data to beprocessed.

FIG. 6 illustrates a process of correcting a first motion picture usinginformation extracted from a second motion picture according to variousembodiments. Although FIG. 6 is illustrated with the sampling mode ofFIG. 5A, various embodiments of the present disclosure may not berestrictive hereto.

Referring to FIG. 6, the 3D conversion module 190 may sample the firstmotion picture at points where the ideal path 610 intersects or adjoinsthe actual path 620, and then may generate the sampled second picture.In this case, the 3D conversion module 190 may correct other frames ofthe first motion picture based on frames 611, 612, and 613 of thesampled second picture.

The 3D conversion module 190 may determine correction parameters for thecase that frames relatively distant from the ideal path 610 move towardthe ideal path 610. Although FIG. 6 is illustrated only in 2D motion,various embodiments of the present disclosure may not be restrictivehereto. For example, correction parameters X, Y, Z, roll, pitch, and yawmay be included therein by 6 DOF.

For example, in the case of sampling the frames 611, 612, and 613intersecting or adjoining the ideal path 610, other frames 621, 622,623, and 624, which are not sampled, may shift to frames 621 a, 622 a,623 a, and 624 a on the ideal path 610. In this case, it may bepermissible to maintain data in areas where the frames correspondthereto even before and after the shift of the frames. Otherwise, it maybe permissible to reflect data, based on the frames 611, 612, and 613which are sampled before and after the shift, to areas which are lost oradded by the shift.

According to various embodiments, during the photographing of the firstmotion picture, although picture information at locations between theframes 622 and 612 may not be present because a user does not move in auniform speed, it may be even permissible to additionally generatepositional data (not shown) between the locations based on data of theframes 612 and 622.

FIG. 7 illustrates a process of correcting a first motion picture basedon feature information of a sampled second picture according to variousembodiments. Although FIG. 7 is illustrated with the sampling mode ofFIG. 5A, various embodiments of the present disclosure may not berestrictive hereto.

Referring to FIG. 7, a 3D conversion module may generate the sampledsecond picture by sampling the first motion picture at points where anideal path 710 intersects or adjoins an actual path 720. In this case,the 3D conversion module 190 may correct other frames based on frames711, 712, and 713 of the sampled second picture.

The 3D conversion module 190 may determine a scaling-down degree ofother frames based on the number of features respectively included inthe sampled frames 711, 712, and 713. For example, in the case that thefeatures are arranged in the order of the frames 711, 712, and 713, aframe area 722 photographed after the frame 712 including a sufficientnumber of features may be converted into a frame area 722 a with thesmallest size, having the largest scaling-down degree reflected thereto.A frame area 723 with a small amount of calculation due to a relativelysmall number of features may be converted into a frame area 723 a withthe largest size. A frame area 721 with a middle number of features maybe converted into a frame area 721 a.

As described above, in the case of correcting the first motion picturebased on features, it may be permissible to maintain the total amount ofcalculation in a uniform level, thereby processing data and generating a3D view in a specific time.

FIG. 8 is a flow chart showing a process of mode change based on asecond motion picture according to various embodiments.

Referring to FIG. 8, at operation 810, a 3D conversion module 190 maysample a first motion picture and then may generate a second motionpicture. A mode of sampling the second motion picture may be performedby using an ideal path or by setting a reference area, thereby resultingin a sampled second picture.

At operation 820, the 3D conversion module 190 may extract spatialinformation such as features, depths for the generated second picture.

At operation 830, the 3D conversion module 190 may determine whether therelative similarity for each frame of the sampled second picture isequal to or higher than a specific value. The 3D conversion module 190may confirm the relative similarity by comparing features of each frameeach other in number, distribution, and similarity.

At operation 840, in the case that the relative similarity is lower thanthe specific value, the 3D conversion module 190 may change aphotographing mode. This is because low relative similarity thereofindicates that the currently photographing first picture is varying to astate which is unsuitable to be converted into a 3D view. The 3Dconversion module 190 may detect additional features, through a changeof a photographing mode, and may correct an error. Additionalinformation about the photographing mode change may be provided throughthe flow shown in FIG. 9.

At operation 850, in the case that the relativity is equal to or higherthan the specific value, extracted spatial information may be reflectedto generation of a 3D view for the first motion picture.

FIG. 9 is a flow chart showing a process of photographing mode changeaccording to various embodiments.

Referring to FIG. 9, at operation 910, a 3D conversion module 190 maybegin to change a photographing mode.

At operation 920, the 3D conversion module 190 may determine whetherthere is a variation of features in a degree equal to or larger than aspecific value between two continuous frames of a second motion picture.For example, in the case that an object includes a shadow area orabruptly varies in feature, the features between two continuous framesmay vary in a degree equal to or larger than a specific value.

At operation 930, in the case the features vary in a degree equal to orlarger than the specific value, the 3D conversion module 190 maydetermine whether a dispersion value of the features between the twoframes varies in a degree equal to or larger than a specific value.

At operation 940, in the case that the dispersion value of the featuresvaries in a degree equal to or larger than the specific value (e.g., thecase that a shadow partly appears in the area of an object), the 3Dconversion module 190 may change a photographing mode to a sensorbooster mode. The sensor booster mode may change at least one of anoperating speed, sensitivity, and exposure time for an image sensor tomake features easily extracted.

At operation 950, in the case that the dispersion value of the featuresdoes not vary in a degree equal to or larger than the specific value(e.g., the case that a photographing area turns to the head, which hasrelatively small feature information, in the visage which has relativelymany feature information), the 3D conversion module 190 may change aphotographing mode to a sensor fusion mode. The sensor fusion mode mayutilize a plurality of sensors.

At operation 960, the 3D conversion module 190 may determine whether therelative similarity of features between the two frames is equal to orhigher than a specific value. In the case that the similarity betweenthe frames is equal to or higher than a specific value, the 3Dconversion module 190 may terminate the process of photographing modechange.

At operation 970, in the case that the relative similarity of featuresis lower than the specific value, the 3D conversion module 190 maychange a photographing mode to a guide mode. The guide mode may guide auser to a photographing direction or a photographing mode.

For example, in the case that a user changes a location or angle of acamera or object over a specific range while horizontally rotating thecamera or object, an electronic device 101 may output a message toreturn the camera or object to its previous location and thereby mayguide the user. Additionally, in the case that a user rotates adirection of a camera or object too fast, the electronic device 101 mayoutput a message to guide the user.

At operation 980, the 3D conversion module 190 may determine whetherthere is an error such as a hole.

At operation 990, in the case that there is an error, the 3D conversionmodule 190 may change a photographing mode to a repair mode. The repairmode may use a preliminarily stored 3D view to compensate an area inwhich the error occurs.

FIG. 10 is a flow chart showing an object centering process according tovarious embodiments.

Referring to FIG. 10, at operation 1010, a 3D conversion module 190 maysample a part of frames from a first motion picture and then maygenerate a sampled second picture.

At operation 1020, the 3D conversion module 190 may extractphotographing positional information of the electronic device 101,spatial information, and reference area information, respective toframes of the sampled second picture. The reference area may bepreliminarily set in a specific size. For example, the reference areamay be set in a specific part which corresponds to 20% of the totalframe area.

At operation 1030, the 3D conversion module 190 may perform an objectcentering process and a view smoothing process for an object. The objectcentering process may set the center of a screen in consideration of thenumber or distribution of extracted features. For example, assuming thatan object from which features are uniformly detected to all locations,the center of a screen may become a geometrical center of a figure(e.g., tetragon) in which features detected in corresponding directionsare confined in the least area. In various embodiments, the 3Dconversion module 190 may even shift an area, which has the most numberof features, to the center (the center of density) of a screen.Otherwise, the 3D conversion module 190 may even shift a preliminarilydesignated object (e.g., visage) to the center of a screen. The viewsmoothing process may lessen inter-frame heterogeneity which arises fromthe object centering process.

Through the object centering process and/or the view smoothing process,the largest cropping area may be extracted. For the object centeringprocess or the view smoothing process, the largest cropping area may bea cropping area in a frame which has the largest part (the rest partexcept an area including an object) to be cropped.

At operation 1040, the 3D conversion module 190 may determine whetherthe largest cropping area is wider than the reference area.

At operation 1050, in the case that the largest cropping area is widerthan the reference area, the 3D conversion module 190 may resetparameters to weakly perform the object centering or view smoothingprocess. Through this operation, the largest cropping area may be scaleddown to extend the remaining area on the object.

During the operation 1060, in the case that the largest cropping area isequal to or smaller than the reference area, the 3D conversion module190 may determine a cropping area and may progress a cropping processtherewith.

According to various embodiments, an image processing method performedin an electronic device may include collecting a first motion picture,extracting spatial information for at least a part of frames amongframes forming the first motion picture, and reflecting the extractedspatial information to generation of a 3D view corresponding to thefirst motion picture.

According to various embodiments, the collecting of the first motionpicture may include obtaining a first 2D image corresponding to a firstdirection of an object, and a second 2D image corresponding to a seconddirection of the object. The extracting of the spatial information mayinclude determining the spatial information based on at least a part ofthe first 2D image and the second 2D image. The determining of thespatial information may include determining a first feature of the first2D image and a second feature of the second 2D image corresponding tothe first feature; and based on at least a part of the first feature andthe second feature, determining a location or direction of the object asat least a part of the spatial information. The spatial information maybe stored together with at least a part of the first 2D image and thesecond 2D image.

According to various embodiments, reflecting to the 3D view generationmay include determining a third direction based on the first directionand the second direction, and generating a 3D image of the object basedon the third direction.

According to various embodiments, the reflecting to the 3D viewgeneration may include correcting a plurality of images, whichcorrespond to a plurality of directions, based on the spatialinformation, and providing the 3D view based on the correcting. Thecorrecting may include generating a virtual ideal path of the electronicdevice, and correcting a location of at least one of a plurality ofimages based on the virtual ideal path and the spatial information.

According to various embodiments, the extracting of the spatialinformation may include determining the part of frames by scaling downor sampling the first image under a specific condition.

According to various embodiments, the extracting of the spatialinformation may include, if an ideal path intersects an actual pathalong which the electronic device moves or the ideal path adjoins theactual path in a specific range, sampling the first motion picture. Theextracting of the spatial information may include extracting acorrection parameter for converting a non-sampled frame of the firstmotion picture into a frame photographed on the ideal path.

According to various embodiments, the extracting of the spatialinformation may include scaling down non-sampled frames of the firstmotion picture based on the number of features included in each framedetermined as the part of frames.

According to various embodiments, the extracting of the spatialinformation may include determining the part of frames based on imagedata included in a reference area of each of frames forming the firstmotion picture. The extracting of the spatial information may includeextracting at least one of information about a feature and a depth foreach frame determined as the part of frames, or information about alocation of the electronic device photographing each frame.

According to various embodiments, the reflecting to the 3D viewgeneration may include changing a photographing mode of the first motionpicture based on the spatial information. The changing of thephotographing mode may include, if a feature between continuous framesforming the part of frames varies in a degree equal to or higher than aspecific value and a dispersion value of the feature varies in a degreeequal to or higher than a specific value, collecting the first motionpicture by a first mode. The first mode may collect a first motionpicture by changing at least one of an operating speed, sensitivity, andan exposure time of an image sensor.

According to various embodiments, the changing of the photographing modemay include, if a feature between continuous frames forming the part offrames varies in a degree equal to or higher than a specific value and adispersion value of the feature varies in a degree lower than a specificvalue, collecting the first motion picture by a second mode. The secondmode may use a plurality of sensors to collect the first motion picture.

According to various embodiments, the changing of the photographing modemay include, if relativity between frames included in the part of framesis lower than a specific value collecting the first motion picture in athird mode guiding a user to a photographing direction or aphotographing speed.

FIG. 11 is an exemplary diagram illustrating screens for an objectcentering process according to various embodiments.

In the case that a user cannot photograph an object while laying theobject on the center of a screen, the 3D conversion module 190 may shiftthe object to the center of the screen through a centering process.

For example, in the case of sampling first to third frames 1110 to 1130,an object may be laid at the center, or even at the left or right sideof a screen in each of sampled frames.

The 3D conversion module 190 may extract photographing positionalinformation of an electronic device 101, spatial information of anobject, and information of a reference area. The reference area may be apart in which features of the object are mainly disposed.

In the case that the largest cropping area is larger than the referencearea, the 3D conversion module 190 may inadvertently remove a part offeatures of an object on the center of the object through the croppingprocess. To prevent this, the 3D conversion module 190 may reset theremaining area and then may allow the edge of the screen to be croppedto remain the object.

The 3D conversion module 190 may perform the object centering process orthe view smoothing process for frames 1110, 1120, and 1130 to obtainframes 1111, 1121, and 1131, respectively. In this case, although thecentering is successful on the center of the screen, features of theobject may be cropped because of that.

The 3D conversion module 190 may control parameters to weakly performthe object centering process or the view smoothing process. Through thisprocedure, the frames 1111, 1121, and 1131 may be respectively convertedinto frames 1112, 1122, and 1132. The frames 1112, 1122, and 1132 arerelatively weakened in effect of the object centering process, butfeatures of the object may be maintained without the cropping.

FIGS. 12A and 12B are exemplary diagrams illustrating screens using 3Dviews according to various embodiments. FIGS. 12A and 12B areillustrated as using 3D views but the various embodiments of the presentdisclosure may not be restrictive hereto.

Referring to FIGS. 12A, a 3D conversion module 190 may generate a 3Dview 1210 for a first motion picture based on spatial information of asecond motion picture. The 3D view 1210 may be generated by utilizingthe spatial information of the second motion picture which is scaleddown or sampled. For example, in the case that the 3D conversion module190 performs 6-DOF calculation for the second motion picture, 3-DOFcalculation for the first motion picture may be performed to simplifythe calculation process. The 3D conversion module 190 may generate the3D view 1210, through a background processing, at the almost same timeof completing the photographing of the first motion picture or whileplaying the photographed result.

In various embodiments, the 3D view 1210 may be used for a background ofthe screen. In the case of setting the 3D view 1210 in the background ofthe screen, it may be permissible to differentiate a plane, on which the3D view 1210 is output, in accordance with a detection state of a motionsensor.

For example, in the case that a user is facing and holding an electronicdevice 1201, the front of the 3D view 1210 may be displayed on thescreen. In the case that a user rotates the electronic device 1210 inthe left and right directions, the 3D view 1210 may be rotated byreflecting a rotating direction and strength thereto.

Referring to FIG. 12B, a 3D view 1220 may be used for unlocking ascreen. For example, in a locking state, the front of the 3D view may bedisplayed on the screen. In the case that a user touches, for unlocking,and swipes the screen from the left to the right, the 3D view 1220 maybe rotated in response to the swiping operation. By the rotation of the3D view 1220, the screen may be unlocked.

FIG. 13 is a block diagram 1300 illustrating an electronic device 1301according to various embodiments.

Referring to FIG. 13, the electronic device 1301 may include, forexample, all or a part of elements of the electronic device 101 shown inFIG. 1. The electronic device 1301 may include at least one of one ormore Application Processors (AP) 1310, a communication module 1320, aSubscriber Identification Module (SIM) card 1324, a memory 1330, asensor module 1340, an input unit 1350, a display 1360, an interface1370, an audio module 1380, a camera module 1391, a power managementmodule 1395, a battery 1396, an indicator 1397, or a motor 1398.

The processor 1310 may drive an Operating System (OS) or an applicationto control a plurality of hardware or software elements connected to theprocessor 1310 and may process and compute a variety of data includingmultimedia data. The processor 1310 may be implemented with aSystem-on-Chip (SoC), for example. According to an embodiment, the AP810 may further include a Graphic Processing Unit (GPU) and/or an imagesignal processor. The processor 1310 may even include at least a part ofthe elements shown in FIG. 13. The processor 1310 may load and processinstructions or data, which are received from at least one of otherelements (e.g., a nonvolatile memory), and then store diverse data intoa nonvolatile memory.

The communication module 1320 may have a configuration same with orsimilar to the communication interface 170 of FIG. 1. The communicationmodule 1320 may include a cellular module 1321, a WiFi module 1323, aBluetooth module 1325, a GPS module 1327, an NFC module 1328, and aRadio Frequency (RF) module 1329.

The cellular module 1321 may provide voice call, video call, a characterservice, or an Internet service through a communication network.According to an embodiment, the cellular module 1321 may performdiscrimination and authentication of an electronic device within acommunication network using a subscriber identification module (e.g., aSIM card) 1324. According to an embodiment, the cellular module 1321 mayperform at least a portion of functions that the processor 1310provides. According to an embodiment, the cellular module 821 mayinclude a communication processor (CP).

Each of the WiFi module 1323, the Bluetooth module 1325, the GPS module1327, and the NFC module 1328 may include a processor for processingdata exchanged through a corresponding module, for example. In someembodiments, at least a part (e.g., two or more elements) of thecellular module 1321, the WiFi module 1323, the Bluetooth module 1325,the GPS module 1327, and the NFC module 1328 may be included within oneintegrated circuit (IC) or an IC package.

The RF module 1329 may transmit and receive, for example, communicationsignals (e.g., RF signals). The RF module 1329 may include atransceiver, a Power Amplifier Module (PAM), a frequency filter, a LowNoise Amplifier (LNA), or an antenna. According to another embodiment,at least one of the cellular module 1321, the WiFi module 1323, theBluetooth module 1325, the GPS module 1327, and the NFC module 1328 maytransmit and receive an RF signal through a separate RF module.

The SIM card 1324 may include, for example, a card, which has asubscriber identification module, and/or an embedded SIM, and includeunique identifying information (e.g., Integrated Circuit Card Identifier(ICCID)) or subscriber information (e.g., integrated mobile subscriberidentify (IMSI)).

The memory 1330 (e.g., the memory 130 of FIG. 1) may include, forexample, an embedded memory 1332 or an external memory 1334. Forexample, the embedded memory 832 may include at least one of a volatilememory (e.g., a dynamic RAM (DRAM), a Static RAM (SRAM), a SynchronousDynamic RAM (SDRAM), etc.), a nonvolatile memory (e.g., a One-timeProgrammable ROM (OTPROM), a programmable ROM (PROM), an Erasable andProgrammable ROM (EPROM), an Electrically Erasable and Programmable ROM(EEPROM), a mask ROM, a flash ROM, a NAND flash memory, a NOR flashmemory, etc.), a hard drive, or Solid State Drive (SSD).

The external memory 1334 may further include a flash drive, for example,a Compact Flash (CF), a Secure Digital (SD), a micro-SD, a mini-SD, anextreme Digital (xD), MultiMedia Card (MMC), or a memory stick. Theexternal memory 1334 may be functionally connected with the electronicdevice 1301 through various interfaces.

The sensor module 1340 may measure, for example, a physical quantity, ordetect an operation state of the electronic device 800, to convert themeasured or detected information to an electric signal. The sensormodule 1340 may include at least one of a gesture sensor 1340A, a gyrosensor 1340B, a barometer pressure sensor 1340C, a magnetic sensor1340D, an acceleration sensor 1340E, a grip sensor 1340F, a proximitysensor 1340G, a color sensor 1340H (e.g., RGB sensor), a biometricsensor 1340I, a temperature/humidity sensor 840J, an illuminance sensor1340K, or an UV sensor 1340M. Additionally or generally, though notshown, the sensor module 1340 may further include an E-nose sensor, anElectromyography Sensor (EMG) sensor, an ElectroEncephaloGram (EEG)sensor, an ElectroCardioGram (ECG) sensor, an infrared (IR) sensor, aniris sensor, or a fingerprint sensor, for example. The sensor module 840may further include a control circuit for controlling at least one ormore sensors included therein. In some embodiments, the electronicdevice 1301 may further include a processor, which is configured tocontrol the sensor module 1340, as a part or additional element, thuscontrolling the sensor module 840 while the processor 1310 is in a sleepstate.

The input unit 1350 may have a configuration similar to the input/outputinterface 150 of FIG. 1. Additionally, the display 160 may partlyperform a function of the input/output unit 1350 in the case that it ismade up with a touch panel. The input unit 1350 may include, forexample, a touch panel 1352, a (digital) pen sensor 1354, a key 1356, oran ultrasonic input unit 1358. The touch panel 1352 may recognize, forexample, a touch input using at least one of a capacitive type, aresistive type, an infrared type, or an ultrasonic wave type.Additionally, the touch panel 1352 may further include a controlcircuit. The touch panel 1352 may further include a tactile layer toprovide a tactile reaction for a user.

The (digital) pen sensor 1354 may be a part of the touch panel 1352, ora separate sheet for recognition. The key 1356, for example, may includea physical button, an optical key, or a keypad. The ultrasonic inputunit 1358 may allow the electronic device 1301 to detect a sound waveusing a microphone (e.g., a microphone 1388), and determine data throughan input tool generating an ultrasonic signal.

The display 1360 (e.g., the display 160) may have the same or similarconfiguration with the display 160, including a panel 1362, a hologramdevice 1364, or a projector 1366. The panel 1362 may include the same orsimilar configuration with the display 160 of FIG. 1. The panel 1362,for example, may be implemented to be flexible, transparent, orwearable. The panel 1362 and the touch panel 1352 may be implemented inone module. The hologram device 1364 may show a three-dimensionalpicture in a space using interference of light. The projector 1366 mayproject light onto a screen to display a picture. The screen, forexample, may be positioned in the inside or outside of the electronicdevice 1301. According to an embodiment, the display 1360 may furtherinclude a control circuit for controlling the panel 1362, the hologramdevice 1364, or the projector 1366.

The interface 1370, for example, may include a high-definitionmultimedia interface (HDMI) 1372, a USB 1374, an optical interface 1376,or a D-sub (D-subminiature) 1378. The interface 1370 may include, forexample, the communication interface 170 shown in FIG. 1. The interface1370, for example, may include a Mobile High-definition Link (MHL)interface, an SD card/MMC interface, or an Infrared Data Association(IrDA) standard interface.

The audio module 1380 may have the same or similar configuration withthe input/output interface 150 of FIG. 1 and may convert, for example,sound and an electric signal in dual directions. At least a part ofelements of the audio module 1380 may be included, for example, in theinput/output interface 150 shown in FIG. 1. The audio module 1380, forexample, may process sound information which is input or output througha speaker 1382, a receiver 1384, an earphone 1386, or a microphone 1388.

The camera module 1391 may be a unit which is capable of taking a stillpicture and a moving picture. According to an embodiment, the cameramodule 1391 may include one or more image sensors (e.g., a front sensoror a rear sensor), a lens, an Image Signal Processor (ISP), or a flash(e.g., an LED or a xenon lamp).

The power management module 1395 may manage, for example, power of theelectronic device 1301. The power management module 1395 may include,for example, a Power Management Integrated Circuit (PMIC), a chargerintegrated circuit, or a battery or fuel gauge. The PMIC may operate inwired and/or wireless charging mode. A wireless charging mode mayinclude, for example, diverse types of magnetic resonance, magneticinduction, or electromagnetic wave. For the wireless charging, anadditional circuit, such as a coil loop circuit, a resonance circuit, ora rectifier, may be further included therein. The battery gauge, forexample, may measure a remnant of the battery 1396, a voltage, acurrent, or a temperature during charging. The battery 1396 may include,for example, a rechargeable battery and/or a solar battery.

The indicator 1397 may have the same or similar configuration with theinput/output interface 150 and may display the specific states of theelectronic device 1301 or a part (e.g., the processor 1310) thereof,e.g., a booting state, a message state, or a charging state. The motor1398 may convert an electric signal into mechanical vibration andgenerate a vibration or haptic effect. Although not shown, theelectronic device 1301 may include a processing unit (e.g., a GPU) forsupporting a mobile TV. The processing unit for supporting the mobileTV, for example, may process media data that is based on the standard ofDigital Multimedia Broadcasting (DMB), Digital Video Broadcasting (DVB),or Media Flow (MediaFlo™).

Each of the above-described elements of the electronic device accordingto an embodiment of the present disclosure may be implemented using oneor more components, and a name of a relevant component may vary with onthe kind of the electronic device. The electronic device according tovarious embodiments of the present disclosure may include at least oneof the above components. Also, a part of the components may be omitted,or additional other components may be further included. Also, some ofthe components of the electronic device according to the presentdisclosure may be combined to form one entity, thereby making itpossible to perform the functions of the relevant componentssubstantially the same as before the combination.

According to various embodiments, an electronic device with a cameramodule, the electronic device may include the camera module configuredto photograph a picture, and a 3D conversion module configured togenerate a 3D view based on the picture, wherein the camera modulebegins to collect a first motion picture, and wherein the 3D conversionmodule generates a 3D view corresponding to the first motion picturebased on spatial information about at least a part of frames forming thefirst motion picture. The electronic device may further include a sensormodule, wherein the 3D conversion module extracts spatial informationabout a second motion picture based on information collected through thesensor module.

FIG. 14 is a block diagram illustrating a program module 1410 accordingto various embodiments.

Referring to FIG. 14, according to an embodiment, the program module1410 may include an operating system (OS) to control resources relevantto an electronic device (e.g., the electronic device 101), and/ordiverse applications driven on the operating system. The operatingsystem may be, for example, android, iOS, windows, symbian, tizen, orbada.

The program module 1410 may include a kernel 1420, a middleware 1430, anAPI 960, and/or an application 1470. At least a part of the programmodule 1410 may be preloaded on an electronic device, or may bedownloadable from an external electronic device.

The kernel 1420 may include, for example, a system resource manager 1421or a device driver 1423. The system resource manager 1421 may performcontrol, allocation, or retrieval of system resources. According to anembodiment, the system resource manager 1421 may include a processmanaging part, a memory managing part, or a file system managing part.The device driver 1423 may include, for example, a display driver, acamera driver, a bluetooth driver, a common memory driver, an USBdriver, a keypad driver, a WiFi driver, an audio driver, or aninter-process communication (IPC) driver.

The middleware 1430 may provide, for example, a function necessary forthe application 1470 in common, or provide diverse functions to theapplication 1470 through the API 1460 to allow the application 1470 toefficiently use limited system resources of the electronic device.According to an embodiment, the middleware 1430 may include at least oneof a runtime library 1435, an application manager 1441, a window manager1442, a multimedia manager 1443, a resource manager 1444, a powermanager 1445, a database manager 1446, a package manager 1447, aconnectivity manager 1448, a notification manager 1449, a locationmanager 1450, a graphic manager 1451, or a security manager 1452.

The runtime library 1435 may include, for example, a library modulewhich is used by a compiler to adding a new function through aprogramming language while the application 1470 is being executed. Theruntime library 1435 may perform input/output management, memorymanagement, or capacities about arithmetic functions.

The application manager 1441 may manage, for example, a life cycle of atleast one application of the application 1470. The window manager 1442may manage a GUI resource which is used in a screen. The multimediamanager 1443 may identify a format necessary for playing diverse mediafiles, and perform an encoding or decoding work for media files by usinga codec suitable for the format. The resource manager 1444 may manageresources such as a storage space, memory, or source code of at leastone application of the application 1470.

The power manager 1445, for example, may operate with a basicinput/output system (BIOS) to manage a battery or power, and providepower information for an operation of an electronic device. The databasemanager 1446 may generate, search, or modify a database which is to beused in at least one application of the application 1470. The packagemanager 1447 may install or update an application which is distributedin a form of package file.

The connectivity manager 1448 may manage, for example, wirelessconnection such as WiFi or Bluetooth. The notification manager 1449 maydisplay or notify an event such as arrival message, promise, orproximity notification in a mode that does not disturb a user. Thelocation manager 1450 may manage location information of an electronicdevice. The graphic manager 1451 may manage a graphic effect that isprovided to a user, or manage a user interface relevant thereto. Thesecurity manager 1452 may provide a general security function necessaryfor system security or user authentification. According to anembodiment, if an electronic device (e.g., the electronic device 101)includes a telephony function, the middleware 1430 may further includesa telephony manager for managing a voice or picture call function of theelectronic device.

The middleware 1430 may include a middleware module to form acombination of diverse functions of the above-described elements. Themiddleware 1430 may provide a specialized module by a kind of OS inpurpose of offering differentiated functions. Additionally, themiddleware 1430 may remove a part of the preexisting elements,dynamically, or add a new element thereto.

The API 1460 may be, for example, a set of programming functions, andmay be provided in a configuration which is variable depending on an OS.For example, in the case that an OS is the android or the iOS, it may bepermissible to provide one API set per platform. In the case that an OSis the tizen, it may be permissible to two or more API sets perplatform.

The application 1470 may include, for example, one or more applicationscapable of providing functions for a home 1471, a dialer 1472, anSMS/MMS 1473, an Instant Message (IM) 1474, a browser 1475, a camera1476, an alarm 1477, a contact 1478, a voice dial 1479, an e-mail 1480,a calendar 1481, a media player 1482, am album 1483, and a timepiece1484, or for offering health care (e.g., measuring an exercise quantityor blood sugar) or environmental information (e.g., atmosphericpressure, humidity, or temperature).

According to an embodiment, the application 1470 may include anapplication (hereinafter, referred to as “information exchangingapplication” for descriptive convenience) to support informationexchange between the electronic device (e.g., the electronic device 101)and an external electronic device. The information exchangingapplication may include, for example, a notification relay applicationfor transmitting specific information to the external electronic device,or a device management application for managing the external electronicdevice.

For example, the information exchanging application may include afunction of transmitting notification information, which arise fromother applications (e.g., the applications for SMS/MMS, e-mail, healthcare, or environmental information), to an external electronic device.Additionally, the information exchanging application, for example, mayreceive notification information from an external electronic device andmay provide the notification information to a user.

The device management application may manage (e.g., install, delete, orupdate), for example, at least one function (e.g., turn-on/turn-off ofan external electronic device itself (or a part of components) oradjustment of brightness (or resolution) of a display) of an externalelectronic device which communicates with the electronic device, anapplication operating in an external electronic device, or service(e.g., call service or message service) provided from an externalelectronic device.

According to an embodiment, the application 1470 may include anapplication (e.g., a health care application) which is assigned theretoin accordance with a property of the external electronic device.According to an embodiment, the application 1470 may include anapplication which is received from an external electronic device.According to an embodiment, the application 1470 may include a preloadedapplication or a third party application which is downloadable from aserver. The titles of the program module 1410 according to theillustrated embodiment may be modifiable depending on kinds of OSs.

According to various embodiments, at least a part of the program module1410 may be implemented in software, firmware, hardware, or at least twoor more combinations among them. At least a part of the program module1410, for example, may be implemented (e.g., executed) by a processor(e.g., the processor 1410). At least a part of the program module 1410may include, for example, a module, a program, routine, a set ofinstructions, or a process for performing one or more functions.

Each of the above-described elements of the electronic device accordingto an embodiment of the present disclosure may be implemented using oneor more components, and a name of a relevant component may vary with onthe kind of the electronic device. The electronic device according tovarious embodiments of the present disclosure may include at least oneof the above components. Also, a part of the components may be omitted,or additional other components may be further included. Also, some ofthe components of the electronic device according to the presentdisclosure may be combined to form one entity, thereby making itpossible to perform the functions of the relevant componentssubstantially the same as before the combination.

The term “module” used for the present disclosure, for example, may meana unit including one of hardware, software, and firmware or acombination of two or more thereof. A “module”, for example, may beinterchangeably used with terminologies such as a unit, logic, a logicalblock, a component, a circuit, etc. The “module” may be a minimum unitof a component integrally configured or a part thereof. The “module” maybe a minimum unit performing one or more functions or a portion thereof.The “module” may be implemented mechanically or electronically. Forexample, the “module” according to various embodiments of the presentdisclosure may include at least one of an Application-SpecificIntegrated Circuit (ASIC) chip performing certain operations, aField-Programmable Gate Arrays (FPGAs), or a programmable logic device,those of which have been known or to be developed in the future.

At least a part of an apparatus (e.g., modules or functions thereof) ora method (e.g., operations) according to various embodiments of thepresent disclosure, for example, may be implemented by instructionsstored in a computer-readable storage medium in the form of aprogrammable module. The instruction, when executed by a processor(e.g., the processor 120) may perform a function corresponding to theinstruction. Such a computer-readable medium may be, for example, thememory 130.

The computer-readable recording medium may include a hard disk, amagnetic media such as a floppy disk and a magnetic tape, an opticalmedia such as Compact Disc ROM (CD-ROM) and a Digital Versatile Disc(DVD), a magneto-optical media (e.g., floptical disk), and hardwaredevices (e.g., ROM, RAM, and flash memory). Also, a program instructionmay include not only a mechanical code such as things generated by acompiler but also a high-level language code executable on a computerusing an interpreter. The aforementioned hardware unit may be configuredto operate via one or more software modules for performing an operationof the present disclosure, and vice versa.

According to various embodiments, a computer-readable recording mediummay include a program collecting a first motion picture, extractingspatial information about at least a part of frames forming the firstmotion picture, and reflecting the extracted spatial information togeneration of a 3D view corresponding to the first motion picture.

A module or a programming module according to various embodiments of thepresent disclosure may include at least one of the above elements, or apart of the above elements may be omitted, or additional other elementsmay be further included. Operations performed by a module, a programmingmodule, or other elements according to an embodiment of the presentdisclosure may be executed sequentially, in parallel, repeatedly, or ina heuristic method. Also, a portion of operations may be executed indifferent sequences, omitted, or other operations may be added thereto.

While the present disclosure has been shown and described with referenceto various embodiments thereof, it will be understood by those skilledin the art that various changes in form and details may be made thereinwithout departing from the spirit and scope of the present disclosure asdefined by the appended claims and their equivalents.

What is claimed is:
 1. An image processing method performed in anelectronic device, the image processing method comprising: collecting afirst motion picture; extracting spatial information for at least a partof frames among frames forming the first motion picture; and reflectingthe extracted spatial information to generation of a 3-Dimensional (3D)view corresponding to the first motion picture.
 2. The image processingmethod of claim 1, wherein the collecting of the first motion picturecomprises: obtaining a first two-dimensional (2D) image corresponding toa first direction of an object, and a second 2D image corresponding to asecond direction of the object.
 3. The image processing method of claim2, wherein the extracting of the spatial information comprises:determining the spatial information based on at least a part of thefirst 2D image and the second 2D image.
 4. The image processing methodof claim 3, wherein the determining of the spatial informationcomprises: determining a first feature of the first 2D image and asecond feature of the second 2D image corresponding to the firstfeature; and based on at least a part of the first feature and thesecond feature, determining a location or direction of the object as atleast a part of the spatial information.
 5. The image processing methodof claim 2, wherein the reflecting to the 3D view generation comprises:determining a third direction based on the first direction and thesecond direction; and generating a 3D image of the object based on thethird direction.
 6. The image processing method of claim 2, wherein thereflecting to the 3D view generation comprises: correcting a pluralityof images, which correspond to a plurality of directions, based on thespatial information; and providing the 3D view based on the correcting.7. The image processing method of claim 6, wherein the correctingcomprises: generating a virtual ideal path of the electronic device; andcorrecting a location of at least one of a plurality of images based onthe virtual ideal path and the spatial information.
 8. The imageprocessing method of claim 1, wherein the extracting of the spatialinformation comprises: determining the at least the part of frames byscaling down or sampling the first image under a specific condition. 9.The image processing method of claim 1, wherein the extracting of thespatial information comprises: if an ideal path intersects an actualpath along which the electronic device moves or the ideal path adjoinsthe actual path in a specific range, sampling the first motion picture.10. The image processing method of claim 9, wherein the extracting ofthe spatial information comprises: extracting a correction parameter forconverting a non-sampled frame of the first motion picture into a framephotographed on the ideal path.
 11. The image processing method of claim9, wherein the extracting of the spatial information comprises: scalingdown non-sampled frames of the first motion picture based on a number offeatures included in each frame determined as the at least the part ofthe frames.
 12. An electronic device with a camera module, theelectronic device comprising: the camera module configured to photographa picture; and a 3D conversion module configured to generate a 3D viewbased on the picture, wherein the camera module begins to collect afirst motion picture, and wherein the 3D conversion module generates a3D view corresponding to the first motion picture based on spatialinformation about at least a part of frames forming the first motionpicture.
 13. The electronic device of claim 12, further comprising asensor module, wherein the 3D conversion module extracts spatialinformation about a second picture based on information collectedthrough the sensor module.
 14. The electronic device of claim 12,wherein the 3D conversion module changes a photographing mode of thefirst motion picture based on the spatial information.
 15. Theelectronic device of claim 14, wherein if a feature between continuousframes forming the at least the part of frames varies in a degree equalto or higher than a first value and a dispersion value of the featurevaries in a degree equal to or higher than a second value, the 3Dconversion module allows the first motion picture to be collected by afirst mode.
 16. The electronic device of claim 15, wherein the firstmode collects the first motion picture by changing at least one of anoperating speed, sensitivity, and an exposure time of an image sensor.17. The electronic device of claim 14, wherein if a feature betweencontinuous frames forming the at least the part of frames varies in adegree equal to or higher than a first value and a dispersion value ofthe feature varies in a degree lower than a second value, the 3Dconversion module allows the first motion picture to be collected by asecond mode.
 18. The electronic device of claim 17, wherein the secondmode uses a plurality of sensors to collect the first motion picture.19. The electronic device of claim 14, wherein if relative similaritybetween frames included in the at least the part of the frames is lowerthan a specific value, the 3D conversion module collects the firstmotion picture in a mode guiding a user to a photographing direction ora photographing speed.
 20. A computer-readable recording medium with aprogram comprising: collecting a first motion picture; extractingspatial information about at least a part of frames forming the firstmotion picture; and reflecting the extracted spatial information togeneration of a 3D view corresponding to the first motion picture.